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compensation pixel values are generated to replace missing data from defective 
pixels. 

Applicants traverse this ground of rejection. The references, viewed 
individually or in combination, do not teach or suggest applicants' advantageous 
method for capturing and processing image information within the meaning of 35 
U.S.C. § 103. 

Applicants 5 claimed method and apparatus are directed to recovering missing 
color data in a two-dimensional color array. The method and apparatus involve the 
application of two one-dimensional non-linear interpolation processes. Generally 
speaking, applicants' method essentially "decouples" the conventional two- 
dimensional process into two one-dimensional processes. The first one-dimensional 
color recovery process generates intermediate second color image data from the 
sampled first color image data and the second one-dimensional color recovery process 
generates the desired third color image data from the second color image data. 

As described in detail in the application (see, for example, page 8, line 28 to 
page 10, line 14) a two-dimensional array of discrete image sensing elements, each of 
which is specifically responsive to one of at least three predetermined colors (e.g., 
red, green and blue) is exposed to image information-bearing illumination to obtain a 
collection of each electronic information signal received from each discrete element. 
The collection of signals forms the raw unprocessed one-color image data from which 
fully recovered third color image data can be derived. 

The first step towards deriving the fully-recovered third color image data is to 
first recover missing color information along a first dimension (e.g., along rows of the 
array) by interpolating the first color image data along the first dimension to provide 
a first-interpolated color data for each of the discrete elements, forming a difference 
channel between the first color image data and the first interpolated color data and 
then applying a one-dimensional non-linear filter on the difference channel and 
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combining with the first color data to obtain the first recovered image data. The 
second color image data comprises a combination of the first recovered image data 
and the first color image data. 

The second step of the method derives fully-recovered third color image data 
from the two color image data by recovering missing color information along a 
second dimension, e.g., along columns of the array. The second color image data is 
obtained by interpolating along the second dimension to provide second interpolated 
data for each of the discrete elements, forming a difference channel between the 
second color image data and the second interpolated data and then applying a one 
dimensional non-linear filter on the difference channel and combining with the second 
color data to obtain the second recovered image data. The third color image data 
comprises a combination of the second recovered image data and the second image 
data. 

The method of applicants is not taught or suggested by the references. The 
primary reference, Ercan et al, describes a digital imaging process which is remote 
from that of applicants. As mentioned by the examiner, Ercan et al teaches steps (a) 
and (b) of applicants' method, which steps are generally known in the art. However, 
also as acknowledged by the examiner, this reference does not relate to the recovery 
of missing image information and therefore does not teach or suggest applicants 5 
claimed method. 

Ercan et al relates to a black and white imaging method and is directed to 
improving the contrast of a digital image to accentuate important image details. Areas 
of the image requiring improvement are identified as those with a narrow range of 
pixel values and having a median value that is significantly different than the median 
pixel value of other parts of the image. Proposed corrections act toward bringing the 
median value of the area more toward the average value and increasing its contrast. 
In another aspect of the invention, two different cameras are used to take pictures at 
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different exposures and image processing is used to provide a composite picture with 
well-exposed detail in all areas of the image. 

The image obtained by Ercan et al is not described as having any missing data 
and the processing described is for the purpose of improving the visibility of existing 
image detail whereas the method of applicants involves the recovery of missing data. 

Heller et al, which has been cited as the secondary reference, does not render 
the rejection any more effective. Heller et al deals with correcting defective pixels on 
image sensing elements. Generally, the method of Heller et al involves using certain 
pixels of the image sensor to obtain the missing information and stores the location of 
the defective pixel and of the corrective pixels on the integrated image sensing device 
itself Two embodiments are described (see, for example, column 8, lines 57 - 65). In 
one embodiment, compensation pixel values are generated by examining the pixel 
values surrounding the defective pixel. In the other embodiment the pixel value of a 
pixel that precedes the defective pixel is used as the compensation pixel value. 

The disclosure of the reference is silent as to the color of the pixels which are 
used to provide the compensation values; however it can be inferred that pixels of the 
same color are used to obtain the compensation values. It should also be noted that 
both embodiments of Heller et al use linear filters and there is no explicit teaching of 
the use of non-linear filters as are used in the method of applicants. Applicants' 
method involves the use of second color data and non-linear filters and the method of 
Heller et al does not. Applicants' method also assumes that data is available at all 
pixels and therefore that defective pixels, such as are addressed by Heller et al do not 
exist or have been previously compensated. 

It is readily apparent that one skilled in the art and knowing of the disclosures 
of the references would not be placed in possession of applicants' advantageous 
claimed method and apparatus. In order to properly support a rejection of the claims 
in an application a reference or references must include teaching(s) which would 
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provide an incentive for those skilled in the art to modify the teaching(s) of the 
references to arrive at the claimed subject matter. Here it is readily apparent that the 
teachings of the references cannot be combined to arrive at the claimed method and 
apparatus of applicants. 

Reconsideration of this ground of rejection and withdrawal thereof are 
respectfully requested. 

2. Claims 2 and 7 have been rejected under 35 U.S. C. § 103(a) as being 
unpatentable over Ercan et al in view of United States Patent Application Publication 
No. US2002/0081019 Al ("Katayama et al"). In support of the rejection the 
examiner has generally asserted that although Ercan et al does not specify pattern- 
wise arrangement of image sensors, Katayama et al does teach providing an image 
sensing apparatus which determines whether the optimum image sensing conditions 
are achieved or not by judging whether there is a predetermined pattern in an image 
sensing field. 

Method claim 2 is dependent on claim 1 and apparatus claim 7 is dependent on 
claim 6. These dependent claims are drawn to a preferred embodiment of applicants' 
invention wherein the discrete sensing elements are arranged in a pattern such that (a) 
no two discrete elements that are contiguous along the first or second dimension are 
specifically responsive to the same one of the at least three predetermined colors and 
(b) no more than one discrete element is contiguously between two discrete elements 
that are specifically responsive to the same one of the at least three predetermined 
colors. 

Claims 2 and 7 include, respectively, all the limitations recited in claims 1 and 
6 and are patently distinguishable over the references for the same reasons described 
above with respect to Ercan et al and, further, because Katayama et al does not teach 
or suggest the embodiments of applicants' invention recited in claims 2 and 7. 
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Katayama et al is directed to image sensing apparatus capable of placing an 
object, whose three dimensional shape is to be generated, under the optimum image 
sensing conditions upon sensing the object from a plurality of image sensing points. 
This reference is not directed to recovering missing color information. 

Reconsideration of this ground of rejection and withdrawal thereof are 
respectfully requested. 

3. Claims 3-5 and 8-10 have been rejected under the first paragraph of 35 
U.S.C. § 1 12 as containing subject matter which was not described in the specification 
in such a way as to reasonably convey to one skilled in the relevant art that the 
inventors, at the time the application was filed, had possession of the claimed 
invention. In support of the rejection the examiner has asserted that applicants should 
specify the specifications and parameters for rank-order filters in claims 3 and 8 and 
claims 4, 5, 9 and 10 should have the complete specifications of red, green and blue 
wavelengths. 

Applicants traverse these grounds of rejection. Claims 3-5 and 8-10 satisfy 
all the requirements for patentability recited in Section 1 12. It is beyond dispute that 
the wavelengths present in the red, green and blue regions of the spectrum are well 
known to those skilled in the art. Thus, the specification describes that in a preferred 
embodiment the three predetermined colors are red, green and blue. This statement, 
without more, is sufficient to satisfy the requirements of the first paragraph of Section 
1 1 2 for this embodiment. 

Rank-order filters are also well known to those skilled in the electronic image 
capture and processing art. In the paragraph bridging pages 14 and 15 applicants have 
taught that a particular class of non-linear filters capable of removing narrow peaks 
and valleys within the color-difference data and which therefore are suitable for use in 
the claimed method and apparatus, are rank-order filters. It is further taught that in a 
preferred embodiment a median rank-order filter is used. A suitable rank-order filter 
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is described in U.S. Patent 4,802,108 which is incorporated by reference in the present 
application. 

There are many, many journal articles which describe rank order filters 
generally and median rank order filters in particular. Enclosed are three such journal 
articles which are intended to be illustrative only of the information available to those 
skilled in the art. The references are: 

1 . "LUM Filters: A Class of Rank-Order-Based Filters for Smoothing 
and Sharpening", by Russell C. Hardie and Charles G. Boncelet, IEEE 
TRANSACTIONS ON SIGNAL PROCESSING, Vol. 41, No. 3, 
March 1993, pp. 1061 - 1076; 

2. " Detail-Preserving Ranked-Order Based Filters for Image 
Processing", by Gonzalo R. Arce and Russell E. Foster, IEEE 
TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL 
PROCESSING, Vol. 37, No. 1, January 1989, pp. 83 - 98; and, 

3. " Multiresolution Decomposition and Adapative Filtering with Rank 
Order Based Filters — Application to Defect Detection — by Phillippe 
Salembier, SIGNAL PROCESSING LABORATORY, Swiss Federal 
Institute of Technology, CH-1015 Lausanne, Switzerland, IEEE, July 
1991, pp. 2389-2392. 



The disclosure of the present application specifically teaches that a preferred 
embodiment of the claimed method and apparatus of applicants employs rank order 
filters and median rank order filters. Given the state of the art with respect to such 
filters, as shown by the illustrative journal articles cited above and U.S. Patent 
4,802,108, it is clearly apparent that applicants, at the time the application was filed 
had possession of the embodiment of their invention wherein rank-order filters are 
employed and that claims 3 and 8 satisfy the requirements for patentability established 
by the first paragraph of Section 112. 

Reconsideration of these grounds of rejection and withdrawal thereof are 
respectfully requested. 
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In summary, the claims have been shown to be proper in form for allowance 
and to recite subject matter which is patentably distinguishable over the references of 
record. Reconsideration of the application and allowance of the claims are in order 
and such actions are respectfully solicited. 
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